Techniques for guaranteeing bandwidth with aggregate traffic

ABSTRACT

Methods, systems, and apparatus guarantee bandwidth for a network transaction. A network is logically organized as a tree having a plurality of nodes. Each node can guarantee service for a network transaction through the network. Each node monitors its traffic and reserves predefined amounts of unused bandwidth with its adjacent node. If a particular node needs additional bandwidth, that node borrows the bandwidth from its adjacent node.

This application is a continuation of U.S. patent application Ser. No. 14/153,846, filed Jan. 13, 2014, which is a continuation of U.S. patent application Ser. No. 11/383,980, filed on May 18, 2006, now issued as U.S. Pat. No. 8,631,151, which is a continuation under 35 U.S.C. 111(a) of International Application No. PCT/CN2003/001141, filed on Dec. 30, 2003, and published in English on Jul. 21, 2005 as International Publication No. WO 2005/067203 A1, all of which are incorporated herein by reference in their entireties.

TECHNICAL FIELD

Embodiments of the present invention relate generally to computer networks, and more particularly to bandwidth management for network traffic.

BACKGROUND INFORMATION

Quality of Service (QoS) is the concept that transmission rates, error rates, and other network transmission characteristics can be measured, improved, and to some extent guaranteed in advance of a network transmission. QoS is a significant concern for high bandwidth networks that regularly transmit large amounts of data such as video, audio, multimedia, and the like. Moreover, QoS is problematic for geographically dispersed networks, such as the Internet, where any single network transaction can span multiple sub-networks through multiple Internet Service Providers (ISPs).

Attempts to provide decent QoS architectures often suffer from scalability issues. That is, independent sub-networks (e.g., ISPs) are required to be too heavily dependent upon one another to produce any viable commercial solution. As soon as independent sub-networks become dependent upon the operational specifics of other sub-networks, they become less scalable and less desirable. When scalability is adequately achieved, the result is usually achieved with overly complex implementation schemes that dramatically decrease network throughput at the expense of providing scalability.

Accordingly a more scalable QoS technique for large geographically disperse networks is needed, where scalability is achieved in a manner that does not significantly impact network throughput and is not overly complex.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is diagram of a network to guarantee service in accordance with one embodiment of the invention.

FIG. 2 is a flow diagram of a method to guarantee network service in accordance with one embodiment of the invention.

FIG. 3 is a flow diagram of a method to manage bandwidth of a guaranteed network service request in accordance with one embodiment of the invention.

FIG. 4 is a diagram of a bandwidth management system in accordance with one embodiment of the invention.

DESCRIPTION OF THE EMBODIMENTS

FIG. 1 is a diagram of a network 100 that guarantees service for network transactions before the transactions are processed within the network 100. The technique is implemented in a computer-accessible medium within processing devices of a network. These devices can be routers, hubs, bridges, switches, gateways, firewalls, proxies, servers, client workstations, and the like. The network 100 is logically represented in the computer-accessible medium as a tree where each branch of the tree is a node. A node is a processing device that participates in a network transaction by routing data packets associated with the network transaction between originating nodes and destination nodes.

The embodiments of this invention provide improved techniques for guaranteeing service betweens node of a network 100. Guaranteeing means that bandwidth availability is assured before a network transaction commences, assuming that the nodes and links of the network 100 remain operational and do not otherwise fail. One of ordinary skill in the art readily appreciates that an absolute assurance that a network transaction will complete within a network 100 is not feasible, since links and nodes can fail abnormally due to hardware or software failures, or links and nodes can fail due to planned maintenance activity.

The network 100 depicted in FIG. 1 includes a variety of Autonomous Systems (AS) that can operate independently of one another. These ASs can be viewed as independent sub-networks, such as ISPs, private networks and the like. The sub-networks can geographically span the entire world. Within each AS, a number of processing nodes are used to directly communicate with other external ASs; these nodes are referred to as edge nodes.

The entire network 100 is logically organized as a tree. In FIG. 1, that tree is inverted, such that the root of the tree is identified by AS Y and the farthest leaf is identified by AS X. The root node of the tree is edge node ID. AS N is a father of children 0, 1, and 2. Moreover, the children are considered brothers to each other. Each AS can itself be considered a sub-tree, having its own internal root node, father node, and brother nodes. Moreover, the children of a father are considered its sons. Thus, father AS N has sons identified as Child 0, Child 1, and Child 2. Moreover, AS N is considered a grandfather of AS X.

A network transaction is a communication between any two or more nodes of the network 100. A node that originates the network transaction is called an originating node. A network transaction transfers data packets from an originating node to a destination node. Thus, the node that the originating node desires to communicate with via a network transaction is referred to as a destination node. A network transaction will include the transfer of one or more network data packets from the originating node through one or more intermediate nodes to the destination node. Thus, the network transaction is associated with a path through the network from the originating node to the destination node. A variety of path generating and dynamic modifying algorithms are well known in the networking arts and readily achievable using existing network architectures and protocols. All such algorithms and architectures are capable of being used with the embodiments of this invention.

The volume of and rate at which network packets are present and sent between any two nodes of the network 100, is referred to as the bandwidth. These data transfers occur over the links of the network 100, the links connect the nodes. Each link can be capable of handling different types of media, different volumes, different rates, and a different number of concurrent sessions of network transactions. The hard and soft limitations of each link are known in advance by each node based on its hardware and software configurations. These limitations can be communicated between nodes using well known and existing networking protocols and technologies.

When an originating node requests a particular network transaction directed to a destination node, the originating node would like to know in advance of commencing the transaction that a sufficient amount of bandwidth will exist within the network 100 in order to process the transaction to the destination node. In embodiments of the present invention, this is achieved with a Bandwidth Conservation Criterion (BCC) calculation. The originating node makes a request for a network transaction to the first processing node defined in the network path. The first processing node identifies the total available bandwidth of the destination node and sums the bandwidth of all outstanding traffic that is destined for the destination node. This calculation is the BCC calculation that guarantees the originating node that bandwidth will exist to satisfy the transaction. IN response to the guarantee, the originating node commences the network transaction through the network 100.

As an example, consider an originating node A that requests a network transaction requiring 10 KB of bandwidth. The transaction is directed to destination node N and is initially requested of initial processing node B. N can have a maximum bandwidth of 128 KB. When A makes the request to B, the current aggregate traffic directed to N is 110 KB.

In the present example, B applies BCC to determine that if the network transaction is guaranteed there will be 120 KB of current traffic directed towards N, which is less than the maximum bandwidth of 128 KB that N can handle at any point in time. Thus, B makes the BCC calculation when it receives the request from A and determines that the network transaction can be guaranteed. The guarantee is then communicated from B to A, and the network transaction commences.

BCC can be defined with the following equations, where the originating node is the node identified in AS X and the destination node is edge node D of AS Y (the root of the tree):

$\quad\left\{ \begin{matrix} {{\sum\limits_{i}\; r_{ij}^{D}} \leq r_{jk}^{D}} \\ {B_{ij}^{X} = {\sum\limits_{y}\; r_{ij}^{y}}} \end{matrix} \right.$

B_(ij) ^(X) is the overall bandwidth of AS X available between edge nodes i and j and r_(ij) ^(D), the corresponding portion of AS X in the tree rooted by edge node D. Thus, if the bandwidth request associated with a network transaction when combined with the total aggregate bandwidth destined for edge node D is less than the total bandwidth that edge node D can handle, the BCC equation holds true and a network transaction can be guaranteed service.

This calculation is a scalable approach, because all that is needed is a calculation that sums existing aggregate traffic that is being directed to a destination node along with the known bandwidth limit associated with the destination node. By aggregate traffic it is meant that all current network traffic that is active in the network and is currently being directed to the destination node. However, at any given point as the network transaction is progressing through the network 100, one or more nodes may not have sufficient bandwidth to handle the network request. Thus, BCC can be augmented to make necessary dynamic adjustments as needed at each processing node associated with a network path of a network transaction, which is guaranteed by the BCC equation.

For example, a network transaction may progress after a guarantee of service from AS m to edge node j′ of AS Y. At this point in time, however, the current bandwidth for the link between node j′ and k (link j′-k) may be at capacity and not capable of handling the network transaction. Without the ability for link j′-k to increase its bandwidth the transaction may fail or become unreasonably delayed. Thus, the static approach of BCC can be augmented with a Closest relation AllocatioN (CLAN) technique, that permits nodes to borrow bandwidth from adjacent (brother, son, father) nodes.

With the CLAN technique, each node of the network 100 monitors the traffic volume occurring with its link to its adjacent nodes, more specifically with its father node. When a particular node notices a reduction in bandwidth occurring with its link to its father node, that reduction is known to the father node (adjacent node). This reserved bandwidth is held by the father node and delivered to other more needing sons of the father when needed.

Thus, in the example, presented above, if the link j′-k can hold 128 KB volume and is at capacity when a network transaction comes along that needs to reach node k, then node k can borrow 64 KB from one or both of its two remaining sons. Node k knows the 64 KB is obtainable from one or its two remaining sons, because if finds the two remaining sons having chunks of available bandwidth to their father node K during processing and request bandwidth from their father node when needed.

For example, consider that the three links to node k each have a total bandwidth capacity of 128 KB and consider further that each son node is configured to manage only 64 KB of capacity at any one time and configured to release and to notify the father node k whenever bandwidth is needed above 64 KB and the father node will be aware when capacity above 64 KB is no longer needed for any given transaction. The father node k then manages this excess bandwidth and when a network transaction moving from link j′-k, in our present example, needs additional bandwidth; the father node k has it in reserve to deliver to the son node j′, because the bandwidth has been previously borrowed from one or both brothers of node j′.

Borrowing can occur between any two adjacent nodes or brother nodes. Moreover, the borrowing need not occur within a single AS. For example edge node j′ connects two different ASs (L and M) to AS N. Thus, edge node j of AS M can borrow from the appropriate edge node of AS L in order to complete a transaction over link j-j′. This occurs through j′, which acts as an adjacent node or father node to both node j and the edge node of AS L that directly connects to node j′.

Thus, with CLAN, adjacent nodes establish policies with one another such that chunks of bandwidth are managed by father nodes on behalf of their sons. These chunks of bandwidth are considered reserves, which must be requested before the reserves can be used from the appropriate father node. Thus, son nodes borrow, via their father, bandwidth that exceeds a pre-negotiated amount; the bandwidth borrowed comes from the brother nodes of a borrowing son node, but is managed by their father node. In a similar fashion father node can borrow from its father within the network 100, such that at any one time a borrowing node may be getting bandwidth from its brothers and from brothers of its father.

Hierarchical relationships between nodes of a network are managed in order to borrow bandwidth with the CLAN technique; the borrowing is needed for a network transaction that is progressing through nodes of a network. That is, a node in need of more bandwidth uses its closest relationship to adjacent nodes to acquire the necessary bandwidth allocation and this borrowing can progress upward through that node's relatives after first borrowing from its closest relation.

Further, the BCC calculation permits a transaction guarantee to be given to an originating node of a network transaction, where that network transaction requires a certain amount of bandwidth through a network 100 in order to reach a destination node. Moreover, as the network transaction progresses through the network 100, if any particular link between nodes lacks sufficient bandwidth to process the transaction, the bandwidth can be temporarily borrowed from adjacent nodes using the CLAN technique. The BCC and CLAN techniques provide a scalable solution to QoS for ASs or independent networks. These techniques are not overly complex and can be implemented within existing network protocols and software designed to calculate the BCC and implement the CLAN technique.

FIG. 2 is a flow diagram of one method 200 to guarantee network service by implementing the BCC and CLAN techniques discussed above with FIG. 1. The method 200 is implemented in a computer accessible medium and processes on each node of a network. The nodes are processing devices in the network which originate, route, and process network transactions through a network. The method 200 can be implemented within each node of the network as software, firmware, and/or via network protocols.

Initially, at 210, a request for a network transaction is received by a first processing node. The request originates from an originating node. The network transaction is associated with a network path which defines one or more routes for network packets associated with the network transaction to traverse through one or more intermediate nodes of the network to a destination node. The first processing node is the first node of that network path.

When the first processing node receives the network request from the originating node, it determines whether the request is acceptable based on the BCC calculation by checking if the aggregate traffic existing in the network which is destined for the destination node plus the bandwidth needed by the network transaction exceeds the destination node's maximum bandwidth. The BCC calculation and the appropriate check are made at 220. If the aggregate traffic plus the bandwidth needed does exceed the destination node's bandwidth limit, then, at 222, the network transaction cannot be guaranteed and the originating node is notified of the same.

However, if at 220, the aggregate traffic plus the bandwidth needed does not exceed the destination node's bandwidth limit, then, at 230, the first processing node guarantees to the originating node that the requested network transaction will have sufficient bandwidth to reach the destination node within the network. In one embodiment, as soon as this guarantee occurs, the total available bandwidth at the destination node is decremented at 240 by the bandwidth which is needed to satisfy the current network transaction.

Of course there are a variety of ways to implement the BCC calculation. One technique would be to have the destination node keep track of traffic headed its way and maintain a current available bandwidth counter, which is constantly and dynamically changing. Another way is to have each node dynamically make queries to other nodes in the network to dynamically calculate the BCC. One skilled in the art will readily appreciate that other techniques can also exist. All such techniques, which resolve the BCC calculation, are intended to be included in the embodiments of this invention.

Once the first processing node guarantees service, the network transaction commences to process through the network to the destination node at 250. In some embodiments, at some point, during the processing of the network transaction, a particular processing node may determine that it actually lacks the necessary bandwidth needed to process the network transaction through a link to a next node of the network path associated with the network transaction, as depicted at 260. Each node of the network dynamically implements the CLAN technique discussed above with FIG. 1 in order to dynamically resolve this problem.

Accordingly, at 270, the particular processing node that lacks sufficient bandwidth borrows the needed bandwidth from an adjacent node in order to process the network transaction. Thus, the father node of the particular processing node manages reserve bandwidth on behalf of the particular processing node and the particular processing node's brother nodes. These nodes pre-establish with one another the amount of excess bandwidth that the father node will be responsible for managing and delivering, if and when excess bandwidth is needed by any of the sons of the father. Additionally, in some embodiments, the father node of a needy son may not have sufficient bandwidth to satisfy a needing son's bandwidth request. In these situations, the father contacts his father (adjacent node to the father and grandfather of the needing son) in order to borrow bandwidth from brothers of the father (grandfather to the needing son). This borrowing continues as needed according to the rules associated with the CLAN technique, as described above in FIG. 1.

The embodiments of method 200 demonstrate how the BCC and CLAN technique can be implemented and processed within nodes of a network in order to provide a scalable QoS from heterogeneous ASs logically organized as a single network. These embodiments are not complex and can be deployed with software processing on each network node that uses conventional network protocols to communicate between nodes.

FIG. 3 is a flow diagram of one method 300 to manage bandwidth of a guaranteed network service request. The method 300 is implemented within each node of a network and is implemented in a computer accessible medium. The method 300 represents processing performed by each node and some interactions occurring between nodes during a network transaction. The method 300 represents embodiments of the CLAN technique, where nodes borrow and manage bandwidth within a network during a network transaction.

At 310, a processing node associated with processing a network transaction through a network to a destination node monitors its own traffic volume. The processing node has previously used policies known to its adjacent nodes (brothers, father, and sons) in order to configure it to monitor traffic at specific predefined levels. When traffic falls below the predefined limit, then this is detected at 320, and predefined amounts of bandwidth that are available are reserved as being unavailable to the processing node at 330.

Those predefined amounts of reserved bandwidth are known to the processing node's father node (adjacent node) at 340. The father node manages a pool of reserved bandwidth on behalf of the processing node and on behalf of brothers of the processing node. Thus, at 350, when another node (brother node of the original processing node) is processing its own network transaction and determines that it needs additional bandwidth to process the network transaction through to the father node, the brother node makes a request at 360 to borrow the needed bandwidth from the father node (adjacent node). The brother has directly borrowed the bandwidth from the father, but indirectly borrowed it from the excess capacity that the brother's siblings had previously deposited with the father for purposes of management.

This bandwidth management and borrowing technique reflects an example implementation of the CLAN technique discussed above with FIGS. 1 and 2. Policies about management associated with bandwidth deposits (e.g., reservations) and withdrawals are communicated between adjacent nodes of the network. Moreover, at 370, each node of the network is capable of resolving the BCC calculation when needed by maintaining techniques for resolving (e.g., calculating) the aggregate bandwidth associated with any particular destination node of a particular network transaction.

Method 300 provides an example implementation of the CLAN technique and maintains the capabilities when needed to perform the BCC calculation. This demonstrates how a heterogeneous network consisting of a plurality of nodes can interact in a scalable fashion with one another to provide QoS for a network transaction.

FIG. 4 is a diagram of one bandwidth management system 400. The bandwidth management system 400 represent an embodiment of the BCC and CLAN technique within a heterogeneous network, where the network includes a plurality of sub-networks identified as ASs. The bandwidth management system 400 is implemented in a computer accessible medium.

The bandwidth management system 400 includes logically representing the heterogeneous network as a network tree 401, wherein branches of the tree 401 can include other sub-trees. The tree 401 can be entirely managed and manipulated by pointers and metadata associated with the attributes and characteristics of the tree 401. The tree 401 includes a plurality of nodes 402 and 403. Each node 402 or 403 assumes a designation as a son, father, and/or brother depending upon its context within the tree to another adjacent node 402 or 403. Thus, a single node 402 or 403 can have multiple designations with respect to adjacent nodes 402 and 403. A node can also have a designation with respect to non-adjacent nodes, such as grandfather, grandson, uncle, and the like.

Each node 402 or 403 also includes its own traffic monitor 402A-403A, bandwidth modifier 402B-403B, and signaling processor 4020-4030. These entities combine both software logic and existing networking protocols to perform the BCC and CLAN techniques.

Thus, the traffic monitor 402A-403A monitors traffic on its respective node 402 or 403 and communicates traffic information to the traffic monitors 402A-403A of its adjacent nodes 402 or 403. This communication and monitoring is useful in resolving the processing associated with the BCC and CLAN techniques. For example, traffic volumes can be aggregated to resolve the BCC calculation for a particular destination node of a particular network transaction. Moreover, the traffic volumes can be used to determine whether to deposit bandwidth with or to withdraw on loan bandwidth from an adjacent node 402 or 403.

The bandwidth modifier 402B-403B communicates with the traffic monitor 402A-403A in order to adjust bandwidth associated with its particular processing node 402 or 403. That is, when the traffic monitor 402A-403A reports bandwidth below a predefined and pre-negotiated amount, the bandwidth modifier 402B-403B can be used to implement the CLAN technique and claim additional bandwidth not being used as reserved bandwidth, which is then deposited with an adjacent node 402 or 403. Conversely, when a node 402 or 403 needs additional bandwidth, the bandwidth modifier 402B-403B can be used to detect this need based on a current transaction and based on reports of bandwidth usage from the traffic monitor 402A-403A in order to request more bandwidth on loan from an adjacent node 402 or 403.

The signaling processor 402C-403C can be used to actually allocate and reallocate need bandwidth or excess bandwidth from an adjacent node 402 or 403. That is, the actual device that permits bandwidth to be redirected to a specified link can be controlled with the signaling processor 4020-4030. In essence, bandwidth is reallocated and throttled up or throttled down over physical links when bandwidth is deposited or withdrawn from an adjacent node 402 or 403. This throttling is the responsibility of the signaling processor 402C-403C.

During operation, the traffic monitor 402A-403A reports traffic on its node 402 or 403 to the adjacent nodes 402 or 403. The bandwidth modifier 402B-403B uses this in combination with existing network requests to modify bandwidth. The signaling processor 4020-4030 detects modified bandwidth and drives the underlying bandwidth device to throttle up and down affected links of the nodes 402 and 403.

The use of the traffic monitor 402A-403A, the bandwidth modifier 402B-403B, and the signaling processor 402C-403C provides a modular event-driven implementation of the BCC and CLAN technique that is scalable across a large heterogeneous network. However, one of ordinary skill in the art appreciates that other implementations and architectures are possible where the functions of the modules are further isolated into more modules or combined into less modules. All such modifications to this architecture that are designed to perform the BCC or CLAN techniques fall within the scope of the embodiments for this invention.

The above description is illustrative, and not restrictive. Many other embodiments will be apparent to those of skill in the art upon reviewing the above description. The scope of embodiments of the invention should therefore be determined with reference to the appended claims, along with the full scope of equivalents to which such claims are entitled.

The Abstract is provided to comply with 37 C.F.R. §1.72(b) requiring an Abstract that will allow the reader to quickly ascertain the nature and gist of the technical disclosure. It is submitted with the understanding that it will not be used to interpret or limit the scope or meaning of the claims.

In the foregoing description of the embodiments, various features are grouped together in a single embodiment for the purpose of streamlining the disclosure. This method of disclosure is not to be interpreted as reflecting an intention that the claimed embodiments of the invention require more features than are expressly recited in each claim. Rather, as the following claims reflect, inventive subject mater lies in less than all features of a single disclosed embodiment. Thus the following claims are hereby incorporated into the Description of the Embodiments, with each claim standing on its own as a separate exemplary embodiment. 

1. (canceled)
 2. A non-transitory computer-readable medium including executable instructions that when executed by a network device performs a method to: receive a network transaction request from a preceding network device to the network device requesting that a network transaction be processed through the network device to a next network device; and reserve an amount bandwidth from at least one adjacent network device that has a network link with preceding network device to ensure the amount of bandwidth when processing the network transaction through the network device to the next network device.
 3. The medium of claim 2 further comprising instructions to request that the next network device set aside the amount of bandwidth for processing the network transaction in advance of processing the network transaction.
 4. The medium of claim 3, wherein the instructions to reserve amount of bandwidth further includes instructions to request that the next network device instruct a further network device to the next network device that is along a path for the network transaction to reserve the amount of bandwidth with that further network device in advance of processing the network transaction.
 5. The medium of claim 2, wherein the instructions to reserve the amount of bandwidth further includes instructions to reserve the amount of bandwidth from more than one adjacent network device.
 6. The medium of claim 2, wherein the instructions to reserve the amount of bandwidth further includes instructions to determine an existing amount of bandwidth in use by the network device when with network transaction is received to determine the amount of bandwidth and identify the at least one adjacent network device.
 7. The medium of claim 2, wherein the preceding network device, the network device, the next network device, and the at least one adjacent network device are network routers.
 8. The medium of claim 2, wherein the instruction to receive further include to identify in the network transaction request a path for processing the network transaction through a network from an originating network device to a destination network device.
 9. The medium of claim 8, wherein the instructions to identify further includes instructions to identify the next network device from the path.
 10. The medium of claim 2, wherein the instruction to receive further include instructions to aggregate pending network bandwidth for identifying existing bandwidth in use by the network device and the at least one adjacent network device when the network transaction request is received.
 11. A network device, comprising: a processor; and a memory; wherein the processor is configured to process executable instructions residing in the memory and when the executable instructions are processed cause the processor to: reserve excess bandwidth to process the network transaction from at least one adjacent network device to the network device.
 12. The network device of claim 11, wherein when the executable instructions are processed the processor further caused to aggregate existing network bandwidth in use by the network device and the at least one adjacent network device to determine available bandwidth to the network device and the at least one adjacent network device.
 13. The network device of claim 11, wherein when the executable instructions are processed the processor further caused to instruct a next network device for the network transaction to reserve an amount of bandwidth to process the network transaction.
 14. The network device of claim 11, wherein when the executable instructions are processed the processor further caused to identify an amount of bandwidth to process the network transaction in response to a network transaction request received from a preceding network device to the network device.
 15. The network device of claim 13, wherein when the executable instructions are processed the processor further caused to determine the excess bandwidth as a difference between available bandwidth for the network device and the at least one adjacent network device when the network transaction request is received at the network device subtracted from the amount of bandwidth.
 16. The network device of claim 15, wherein the network device connected through network links to the preceding network device, the at least one adjacent network device, and a next network device that is to process the network transaction.
 17. The network device of claim 11, wherein network device is one of: a router, a hub, a bridge, a switch, a gateway, a firewall, a proxy, and a server.
 18. A system, comprising: a plurality of network devices connected through network links; wherein each network device configured to: guarantee an amount of bandwidth for processing a network transaction through the network links in advance of processing the network transaction, and borrow at least some of the amount of bandwidth from an adjacent network device to that network device.
 19. The system of claim 18, wherein each network device is further configured to: aggregate existing bandwidth available to that network device and that network device's adjacent network device to determine a borrow amount of bandwidth to request from that adjacent network device.
 20. The system of claim 18, wherein each network device is further configured to identify a next network device from the plurality of network devices from a path of the network transaction.
 21. The system of claim 20, wherein the plurality of network devices are one or more of: a router, a hub, a bridge, a switch, a gateway, a firewall, a proxy, a server, and a client workstation. 